Skip to content

When intuition fails, how to use probability, choice, and reason to find the real answers.

Much of our thinking is flawed because it is based on faulty intuition. But by using the framework and tools of probability and statistics, we can overcome this to provide solutions to many real-world problems and paradoxes. Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.

Woman waiting at a bus stop

When it comes to situations like waiting for a bus, our intuition is often wrong.

Imagine, there’s a bus that arrives every 30 minutes on average and you arrive at the bus stop with no idea when the last bus left. How long can you expect to wait for the next bus? Intuitively, half of 30 minutes sounds right, but you’d be very lucky to wait only 15 minutes.

Say, for example, that half the time the buses arrive at a 20-minute interval and half the time at a 40-minute interval. The overall average is now 30 minutes. From your point of view, however, it is twice as likely that you’ll turn up during the 40 minutes interval than during the 20 minutes interval.

This is true in every case except when the buses arrive at exact 30-minute intervals. As the dispersion around the average increases, so does the amount by which the expected wait time exceeds the average wait. This is the Inspection Paradox, which states that whenever you “inspect” a process, you are likely to find that things take (or last) longer than their “uninspected” average. What seems like the persistence of bad luck is simply the laws of probability and statistics playing out their natural course.

Once made aware of the paradox, it seems to appear all over the place.

For example, let’s say you want to take a survey of the average class size at a college. Say that the college has class sizes of either 10 or 50, and there are equal numbers of each. So the overall average class size is 30. But in selecting a random student, it is five times more likely that he or she will come from a class of 50 students than of 10 students. So for every one student who replies “10” to your enquiry about their class size, there will be five who answer “50”. The average class size thrown up by your survey is nearer 50, therefore, than 30. So the act of inspecting the class sizes significantly increases the average obtained compared to the true, uninspected average. The only circumstance in which the inspected and uninspected average coincides is when every class size is equal.

We can examine the same paradox within the context of what is known as length-based sampling. For example, when digging up potatoes, why does the fork go through the very large one? Why does the network connection break down during download of the largest file? It is not because you were born unlucky but because these outcomes occur for a greater extension of space or time than the average extension of space or time.

Once you know about the Inspection Paradox, the world and our perception of our place in it are never quite the same again.

Another day you line up at the medical practice to be tested for a virus. The test is 99% accurate and you test positive. Now, what is the chance that you have the virus? The intuitive answer is 99%. But is that right? The information we are given relates to the probability of testing positive given that you have the virus. What we want to know, however, is the probability of having the virus given that you test positive. Common intuition conflates these two probabilities, but they are very different. This is an instance of the Inverse or Prosecutor’s Fallacy.

The significance of the test result depends on the probability that you have the virus before taking the test. This is known as the prior probability. Essentially, we have a competition between how rare the virus is (the base rate) and how rarely the test is wrong. Let’s say there is a 1 in 100 chance, based on local prevalence rates, that you have the virus before taking the test. Now, recall that the test is wrong one time in 100. These two probabilities are equal, so the chance that you have the virus when testing positive is 1 in 2, despite the test being 99% accurate. But what if you are showing symptoms of the virus before being tested? In this case, we should update the prior probability to something higher than the prevalence rate in the tested population. The chance you have the virus when you test positive rises accordingly. We can use Bayes’ Theorem to perform the calculations.

In summary, intuition often lets us down. Still, by applying the methods of probability and statistics, we can defy intuition. We can even resolve what might seem to many the greatest mystery of them all – why we seem so often to find ourselves stuck in the slower lane or queue. Intuitively, we were born unlucky. The logical answer to the Slower Lane Puzzle is that it’s exactly where we should expect to be!

When intuition fails, we can always use probability and statistics to look for the real answers.

Leighton Vaughan Williams, Professor of Economics and Finance at Nottingham Business School. Read more in Leighton’s new publication Probability, Choice and Reason.

In the wrong hands, vaccination statistics can prove deadly. Simpson’s Paradox shows why.

There has been much discussion of late about data published on 1 November, 2021, by the Office for National Statistics (ONS). It is titled ‘Deaths involving COVID-19 by vaccination status, England: deaths occurring between 2 January and 24 September 2021’.

https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/bulletins/deathsinvolvingcovid19byvaccinationstatusengland/deathsoccurringbetween2januaryand24september2021#deaths-by-vaccination-status-england-data

The raw statistics show death rates in England for people aged 10 to 59, listing vaccination status separately. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/deathsbyvaccinationstatusengland

Counter-intuitively, these statistics show that the death rates for the vaccinated in thus age grouping were greater than for the unvaccinated. These numbers have since been heavily promoted and highlighted on social media by anti-vaccine advocates, who use them to argue that vaccination increases the risk of death. 

The claim is strange, though, because we know from efficacy and effectiveness studies that COVID-19 vaccines offer strong protection against severe disease. For example, the efficiency and effectiveness of the Pfizer-BioNTech vaccine has been shown to be well over 90% in this regard in the most recent studies.  https://www.yalemedicine.org/news/covid-19-vaccine-comparison

Vaccine efficacy of 90% means that you have a 90% reduced risk compared to an otherwise similar unvaccinated person, based on controlled randomised trials, while vaccine effectiveness refers to real-world outcomes. On either measure, vaccines work very well indeed.

So, what’s going on here?

Well, closer inspection of the ONS report reveals that over the period of the study, from January to September 2021, the age-adjusted risk of death involving COVID-19 was 32 times greater among unvaccinated people compared to fully vaccinated people. But hold on! How can we square this with the data from the table listing death rates of those aged 10 to 59 by vaccination status?

For the answer we turn to a classic statistical artefact known as Simpson’s Paradox, which seems to pop up and create misleading conclusions all over the place. https://leightonvw.com/2019/02/14/what-is-simpsons-paradox-and-why-it-matters/

It is a consequence of the way that data is presented. 

Essentially, Simpson’s Paradox can arise when observing a feature of a broad, widely drawn group, where there is an uneven distribution of the population within this group, for example by age or vaccination status. Ignorance of the implications of Simpson’s Paradox can generate misleading conclusions, which can be, and in this case are, verydangerous.

The paradox in these particular ONS statistics arises specifically because death rates increase dramatically with age, so that at the very top end of this age band, for example, mortality rates are about 80 times as high as at the very bottom end. A similar pattern is observed between vaccination rates and age. For example, in the 10 to 59 data set more than half of those vaccinated are over the age of 40. 

Those who are in the upper ranges of the wide 10 to 59 age band are, therefore, both more likely to have been vaccinated and also more likely to die if infected with COVID-19 or for any other reason, and vice versa. Age is acting, in the terminology of statistics, as a confounding variable, being positively related to both vaccination rates and death rates. Put another way, you are more likely to die in a given period if you are older and you are also more likely to be vaccinated if you are older. It is age that is driving up death rates not the vaccinations. Without the vaccinations, deaths would be hugely greater from COVID-19. 

So, what if we divide the 10 to 59 group into smaller age groups?

If we break down the band into narrower age ranges, such as 10 to 19, 20 to 29, 30 to 39, 40 to 49, and 50 to 59, we find that the counter-intuitive headline finding immediately disappears. In each age band, the death rates of the vaccinated are vastly lower than those of the unvaccinated. This also applies in the higher age bands – 60 to 69, 70 to 79, and 80 plus.

Basically, unvaccinated people are much younger on average, and therefore less likely to die. 

Yet there are those out there who are more than happy to use these statistics to mislead. The consequence is that many who would otherwise choose to be vaccinated might refuse to do so. In truth, the age-adjusted risk of deaths involving coronavirus (COVID-19) over the first nine months of this year was in fact 32 times greater in the unvaccinated than the fully vaccinated. This is a hugely important statistic, and we must not let statistical manipulation be used to obscure this critical information.The lives of countless people really do depend on us exposing this truth. 

Leighton Vaughan Williams, Professor of Economics and Finance at Nottingham Business School. https://www.ntu.ac.uk/staff-profiles/business/leighton-vaughan-williams

Read more in Leighton’s new publication, Probability, Choice, and Reason. https://www.amazon.co.uk/Probability-Choice-Leighton-Vaughan-Williams-ebook/dp/B09DPTVFFR/ref=sr_1_2?keywords=probability+choice&qid=1638207631&qsid=262-7509985-0691032&sr=8-2&sres=3540542477%2C0367538911%2C1294977482%2C1108713505%2C1138715336%2C0521747384%2C0387715983%2C3030486001%2C1444333429%2CB07KC98Z3C%2C0071381562%2C0631183221%2C0816614407%2C1848722834%2C3319820346%2CB07SZLGZYH&srpt=ABIS_BOOK

How to spot a faker!

In a fascinating article published in the New York Times, Malcolm Browne relates how Dr. Theodore Hill would ask his mathematics students to go home and either toss a coin 200 times and record the results, or else pretend that they had done so. Either way, he would ask them to produce for him the results of their (real or imaginary) coin-tossing experiment.

Dr. Hill’s purpose in this experiment was to show just how difficult it is to fake data convincingly. It just isn’t that easy to make up a random sequence. Based on this knowledge, he would astound his students by almost unerringly picking out the fakers from the tossers!

One of the ways he would do this would be to spot how many times heads or tails would be listed six or more times in a row. In real life, this occurrence is overwhelmingly probable in 200 coin throws. To most of his students this long a sequence is counter-intuitive, an example of what is often termed the Gamblers’ Fallacy, i.e. the erroneous perception that independent random sequences will balance out over time, so that for example an extended sequence of heads is more likely to be followed by a tail than a head. The fakers, susceptible to the Fallacy, are thus easily exposed. Ordinary people, even mathematics students, simply can’t help introducing patterns into what is random noise.

This is an example of a broader analysis which is usually referred to a Benford’s Law, which essentially states that if we randomly select a number from a table of real-life data, the probability that the first digit will be one particular number is significantly different to it being a different number. For example, the probability that the first digit will be a ‘1’ is about 30%, rather than the intuitive 10%, which assumes that all digits are equally likely. In particular, Benford’s Law applies to the distribution of leading and trailing digits in naturally occurring phenomena, such as the population of different countries or the heights of mountains. For example, choose a paper with a lot of numbers and circle the numbers that occur naturally, such as stock prices. So lengths of rivers lakes could be included, but not artificial numbers like telephone numbers. 30% or so of these numbers will start with a 1, and it doesn’t matter what units they are in. So the lengths of rivers could be denominated in kilometres, miles, feet, centimetres, without it making a difference to the distribution frequency of the digits. 

The empirical support for this proportion can be traced to the man after whom the Law is named, physicist Dr. Frank Benford, in a paper he published in 1938, called ‘The Law of Anomalous Numbers’. In that paper he examined 20,229 sets of numbers, as diverse as baseball statistics, the areas of rivers, numbers in magazine articles and so forth, confirming the 30% rule for number 1. For information, the chance of throwing up a ‘2’ as first digit is 17.6%, and of a ‘9’ just 4.6%. The same principle applies to trailing (i.e. last) digits. It’s a great way, therefore, of checking the veracity of receipts. If, for example, there is an unusual number of trailing digit ‘7’s, there’s a decent chance that the figures are cooked.

To explain the basis of Benford’s Law, take £1 as a base. Assume this now grows at 10% per day.

£1.10, £1.21, £1.33, £1.46, £1.61, £1.77, £1.94, £2.14, £2.35, £2.59, £2.85, £3.13, £3.45, £3.80, £4.18, £4.59, £5.05, £5.56, £6.11, £6.72, £7.40, £8.14, £8.95, £9.84, £10.83, £11.92, £13.11, £14.42, £15.86, £17.45, £19.19, £21.11, £23.22, £25.50, £28.10, £30.91, £34.00, £37.40, £41.14, £45.26, £49.79, £54.74, £60.24, £72.89, £80.18, £88.20, £97.02 …

So we see that the numbers stay a long time in the teens, less in the 20s, and so on through the 90s, and this pattern continues through three digits and so forth. Benford noticed that the probability that a number starts with n = log (n+1) – log (n).

NB log10 1 = 0; log10 2 = 0.301; log10 3 = 0.4771 … log10 10 = 1.

Leading digit                                                        Probability

      1                                                                 30.1%

      2                                                                 17.6%

      3                                                                 12.5%

      4                                                                 9.7%

      5                                                                 7.9%

      6                                                                 6.7%

      7                                                                 5.8%

      8                                                                 5.1%

      9                                                                 4.6%

 

Tax authorities are alert to this, or should be, which should make fraudulent activity just that little bit easier to detect, especially when the fraudster is unaware of the Benford distribution. For all right-minded citizens, we can call that Benford’s Bonus.

Links:

http://www.rexswain.com/benford.html

http://www.jstor.org/pss/984802

 

How to distinguish random sequences from contrived sequences.

Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.

Ask someone to toss a fair coin 32 times. Which of the following rows of coin toss patterns is more likely to result if they actually do toss the coins and record them accurately, and which is likely to be the fake?

HTTHTHTTHHTHTHHTTTHTHTTHTHHTTHHT

OR

HTTHTHTTTTTHTHTTHHHHTTHTHTHHTHHT

 In both cases, there are 15 heads and 17 tails.

But would we expect a run (r) of five Heads or a run of five tails in the series, where r is the length of the run?

The chance of five heads = (1/2) to the power of r = (1/2) to the power of 5 = 1/32. But there are 28 opportunities for a run of five heads in 32 tosses. Same for a run of five tails.

A good rule of thumb is that when N (the number of opportunities for a run to take place) x (1/2 to the power of r) equals 1, it is likely that a run of length, r, will appear in the sequence. So, a run of length r is likely to appear when N = 2 to the power of r.

In the case of 32 coin tosses, with 28 possible runs of length five, N (28) is almost equal to 2 to the power of 5 (32). So a run of five heads (or of tails) is likely if a fair coin is tossed randomly 32 times in a row, and a run of four is almost certain.

Now look at the series of coin tosses above. The first series of 32 coin tosses has no run of heads (or tails) longer than three. The second series has a run of five tails and of four heads.

It is very likely indeed, therefore, that the second series is the genuine one, and the first one is the fake.

Appendix

Probability of 5 heads in a row = 1/32.

Probability of NOT getting 5 heads in a row from a particular run of 5 coin tosses = 31/32

Chance of NOT getting 5 heads in a row from 28 runs of five coin tosses = (31/32) to the power of 28 = 41.1%.

Therefore, the probability of getting 5 heads in a row from 28 runs of five coin tosses = 58.9%.

Similarly for tails.

The Probability of 5 heads OR 5 tails in a row = 1/32 + 1/32 = 1/16

Probability of NOT getting 5 heads OR 5 tails in a row from a particular run of 5 coin tosses = 15/16

Chance of NOT getting 5 heads OR 5 tails in a row from 28 runs of five coin tosses = (15/16) to the power of 28 =16.4%.

Therefore, the probability of getting 5 heads OR 5 tails in a row from 28 runs of five coin tosses = 83.6%

Probability of 4 heads in a row = 1/16.

Probability of NOT getting 4 heads in a row from a particular run of 4 coin tosses = 15/16

Chance of NOT getting 4 heads in a row from 29 runs of four coin tosses = (15/16) to the power of 29 = 15.4%.

Therefore, the probability of getting 5 heads in a row from 28 runs of five coin tosses = 84.6%.

Similarly for tails.

Probability of 4 heads OR 4 tails in a row = 1/16 + 1/16 = 1/8

Probability of NOT getting 4 heads OR 4 tails in a row from a particular run of 4 coin tosses = 7/8

Chance of NOT getting 4 heads OR 4 tails in a row from 29 runs of four coin tosses = (7/8) to the power of 29 = 2.1%

Therefore, the probability of getting 4 heads OR 4 tails in a row from 29 runs of four coin tosses = 97.9%

Exercise

When Nasser Hussain was England cricket captain during 200-01, he lost all 14 coin tosses in the international matches he captained. Given that he captained England in all international matches about a hundred times, what was the probability that he would face this long a losing streak during his captaincy?

The Problem with Teddy – A Short Story

The Problem with Teddy – A Short Story

By Leighton Vaughan Williams

It was difficult to argue with Teddy. Whether he was right or wrong, I could never be sure. But I was sure of one thing, that if he said something was going to happen, it did. That was the problem with Teddy. He never got it wrong. And soon that would become a problem for me.

 

Chapter 1
The problem with language

A dog can expect its master. That is certainly true. But it can’t expect its master next Tuesday,” said Teddy. “Why not?” I asked. “Because a dog has no concept of time?” “No,” responded Teddy, “it is because a dog has no concept of language.” “So can a lion expect a meal when it sees its wounded prey?” I enquired. “You could ask it,” he said, “but you would never understand the answer. Because even if a lion had language, it would be no language we could ever understand.”

“You see,” said Teddy, “language is how we experience the world, as well as the way that we choose to represent it.”

“So language represents the boundaries of what we can know?”, I asked. “You have said it,” he exclaimed. “In clear, plain language.” “This doesn’t mean that nothing exists that can’t be expressed in language, only that it is outside the limits of our philosophy. “There are more things in heaven and earth than are dreamt of in your philosophy,” I offered. “Yes, than in all our philosophies”, he assured me. “But we can never use philosophy to find or explain them.”

“Can you be sure of that?”, I asked. “There is no way to verify that. And if a statement can’t be verified it is meaningless. That’s the test of a meaningful statement.” For a brief moment I felt clever.

“Why do you say that an unverifiable statement is meaningless?’” Teddy asked me. “In that case your own statement is meaningless.”

It was difficult to argue with Teddy. Whether he was right or wrong, I could never be sure. But I was sure of one thing, that I had never shown him to be wrong. Not to myself or to anyone else. That was the genius of Teddy, but it was also the problem.

 

Chapter 2
The problem with probability

“Even a double-headed coin can come down tails,” said Teddy when he entered our shared workspace, displaying his particularly sprightly gait. And his tap, tap, tap of ebony stick. Now, he didn’t need the walking stick. But he did like to tap, tap, tap it along the floor. That was another problem with Teddy.

“I don’t see how a double-headed coin can come down tails,” came my instant riposte. “It’s all about probability,” he said. “It’s a very low probability, but in the quantum universe, a double-headed coin can definitely come down tails.” I assumed he was right, but I couldn’t see how.

He read a lot, and was proud of what he’d learned. “The man who DOES NOT read the great thinkers has no advantage over the man who CAN NOT read them,” Teddy once told me. “The same goes for a woman,” I said, trying to sound enlightened. I liked to sound enlightened in front of Teddy. I don’t know why. I never did. Even when I said something that I thought made some kind of sense. A glance from Teddy always made that abundantly clear.

But I did admire Teddy’s uncanny ability to distinguish what was going to happen from what was not. He had the gift of what some call prescient foresight but what others might call knowing a sure thing.

You see, when Teddy said something would happen, it happened. Like when he called double-six on the pair of dice I had brought from home. That’s a 35 to 1 chance, logic told me, but in my belly I knew it would happen. I knew that as soon as Teddy said six-six. And six-six it was. I guess you could call it a trick, or you could call it magic. I don’t know about that, but I did now something for sure. If Teddy said it would happen, it would. Never bet against Teddy. That was my watchword. Until I did.

 

Chapter 3
The problem with wagers

“It’s usually best to back the favourite”, I told Teddy. “I had read it in a book. A book by an expert.” “That’s true if you’re talking probabilities,” said Teddy. So now I knew it was true. “But if you know something is going to happen, that doesn’t apply,” he said.

And so we went on Sunday night to the casino, at the insistence of the man who knew when things would happen. We never met at the weekend, but today there was a reason, said Teddy. He knew I would win.

“Let’s play roulette,” he said. And produced a wad of notes, a very big wad of notes. “”Red or black,” he asked. “You choose.”

I chose black. “I would choose red,” he said. “It’s your money,” I said. “No, it’s not,” he replied. “It’s yours now, a thousand pounds, to lose, to double, or to keep.”

“Can I just keep the thousand pounds?” I asked, and not risk it on red, or black. It was a joke, of a kind. Teddy was not a generous man, and certainly not generous enough to gift me a grand. And to me it was a lot of money, money I needed to live.

“You’ve struck lucky in the quantum world,” said Teddy. “The thousand is yours. To keep or to spin. I say red, and I say it’s a sure thing.”

“One spin of the wheel, for the lot, or take it home. Your choice.”

Now, when Teddy said something would happen, it did. And he was saying it was going to be red. But my common sense told me that Teddy could not know. The wheel had not yet even started to spin.

“I’ll keep it,” I declared. A thousand pounds. “OK, cash it in,” he said. “It’s yours.” I protested – what if we share it, I said? But he declined. Teddy didn’t need the money. Knowing what would happen had already made him a rich man. And he was not the kind to share. “Good night,” he said, and tap, tap, tapped off into the gathering twilight.

So to the next day, and I asked him how he knew the ball would have landed on red. “I knew we’d never find out,” said Teddy. “Because I knew you’d never wager a thousand pounds on the spin of the wheel.” “But what if I had spun the wheel?” I asked. “Then you would have won,” he said. “A universe in which you would spin that wheel is a universe in which you would be sure to win.” I thought I understood what he meant.

 

Chapter 4
The problem with money

“Does it make you happy, knowing what’s going to happen?” I asked. “Isn’t it a burden?” “I don’t always know what’s going to happen,” he corrected me. “But when I know for sure that something will happen, it does,” he said. “It’s not at all the same thing.”

“But that’s enough to make you a lot of money,” I said. “Knowing some things for sure that others think are unsure has made you so much money.”

And so he told me the tale of Thales, the Greek philosopher, who made his fortune by the application of modern day principles of analysis to ancient day Greece. The story involved forecasts and finance and options on olive presses. I honestly can’t recall all the details. But Teddy could. “Which shows,” he concluded, “that it is much easier for a philosopher to become rich than for a rich man to become a philosopher. But the ambitions of philosophers are of another kind.” It was clear he was talking about himself.

As for me, I just wanted to be rich like Teddy. I knew I would never be as wise.

But all of his great knowledge, great insight, great wisdom – was a burden to him? He seemed to read my mind.

“Great wisdom does not necessarily bring great happiness,” was his now detached observation. “Nor does great riches.”

“So maybe I’m better off being ignorant old me,” I said. “Just seeking the simple things in life, and enough money to enjoy them.”

He shook his head now, disapprovingly.

“Which is better?”, he asked me, “to be a human being dissatisfied or a pig satisfied, to be Socrates dissatisfied or a fool satisfied?” He was quoting one of the great philosophers again. I could tell that by the way he spoke his syllables. But I didn’t really understand the question, let alone the answer. That, I am afraid, was the problem with me.

 

Chapter 5
The problem with cars

We shared coffee and lunch that day, accompanied by the walking stick, the shiny ebony walking stick. I plucked up courage to ask him about the walking stick, why it accompanied him wherever he walked. “This is not a walking stick,” he replied. I did not ask again.

“So what if I told you that I am sure you will be knocked down by a car tomorrow?”, he now asked me.

“You can’t be sure of that,” I said. “I might not go anywhere near a car.” I suspected he was joking. Not a pretty joke, but Teddy and good taste didn’t always see eye to eye.

He reminded me that there was no way of reaching the office without crossing a road. “I’ll be extra careful,” I said.

“You will be knocked down by a car tomorrow,” he repeated, ” and you will be crippled for life.”

He was deadly serious and now I was scared, because when Teddy knew that something was going to happen, it always did.

“It can’t be inevitable,” I said. “What if I don’t even step outside my front door?” “You won’t do that,” he replied. “You are too curious to see if I’m right.” “Nobody’s that curious,” was my instant response. But I was, because I couldn’t see how he could know this. It was like predicting where the roulette ball would land before the wheel even started spinning. I told him so. “Or like predicting six-six on the dice,” he said. I shuddered – and suddenly felt cold.

How could he know? Had he heard of a plot to harm me? Did he know people who knew? Or was he planning to harm me himself. But if so, why warn me? I could make no sense of the problem, no way through the maze. What would Socrates make of this, I wondered. And what advice would he have for the fool?

I asked Teddy for evidence, for proof. He offered none. He said he knew but said he could not explain. Not to me. He gave no reason, but this told me nothing, because he never did. He never told me how he knew that something would happen, but I knew that it always did.

I turned to close friends, close family. Ignore it. Play safe. He’s just trying to frighten you. Maybe he knows something. A mix of opinions, but nothing to help. Not one of them knew Teddy, nor his ebony stick. And not one of them knew that when Teddy knew something, he knew it for sure.

That was the problem with Teddy. And now it had become a very real problem for me.

Chapter 6
The problem with fate

The day wore on and soon a decision had to be made. A choice to make. A choice between the evidence of my experience, that Teddy was never wrong, or my experience of the evidence, of which there was none. I asked Teddy one last time before we retired to our separate homes. Should I stay home all the next day, or should I brave life’s fate? Could I change destiny?

“All fates are possible,” said Teddy, “but the universe where you will come to no harm is not the universe in which you currently live.” I was thinking back now to that spin of the roulette wheel. In a universe where I spun the wheel, I felt sure I would have won. I chose not to. But I could have done. Surely this meant that life’s events were not pre-destined, written in stone and waiting to simply unfold. I could do something about it. I could have spun that wheel. But that would have been a different universe, where everything would be different. Would it even be me on that universe? I wanted to go back, to ask myself to spin that wheel. But I could never meet myself, because yesterday I was a different person, as are we all. We can never go back and meet ourselves, only meet ghostly shadows of who we were, shadows that made us what we are and who we might have been.

I no longer saw things as they were, asking why. I saw things now as they might be, asking why not.

“I can change the world,” I told Teddy. “I can spin that wheel.”

“Yes, we can change our destinies,” he said. “We have the freedom of will to choose.”

It was approaching six and the caretakers came to shut up the building. It was not the perfect arrangement, but it suited us.

He picked up his ebony stick and set off, with his usual jaunty gait. “You are quite the philosopher now,” he called back, “I’ll see you the day after tomorrow.”

“But …” I started to say. He was gone already. That was the problem with Teddy. Always too quick on that stick.

Chapter 7
The problem with thinking

I woke up at dawn next morning and thought of the double-headed coin that might come down tails. But I knew that I could do nothing about that. The quantum world was out of my control.

But some things were within my control, and one was the choice of whether to change life’s plan, to spin the wheel, to change the course of fate.

This could mean staying home, behind closed doors, away from the rush of traffic. This is what it meant to Teddy. But this is not what it meant to me.

Teddy saw things as they were, and he saw things that would be. I now saw things differently. I saw a world as it might be. Where I had the choice to use reason and faith and hope. To conquer fear, on my own terms.

But reason told me that Teddy’s foresight of my fate was not to be overlooked lightly. Teddy didn’t make that kind of mistake.

But Teddy’s universe wasn’t the one I had to inhabit. I could change my destiny. I could stay home, shuttering out the summer day. But I was becoming a philosopher. And the ambitions of philosophers are of another kind.

“A dog can expect its master, but it cannot expect its master next Tuesday,” Teddy had once explained. I thought of that now as I realised that Teddy was not expecting me today. I had become a philosopher, a thinker. Teddy would soon see.

So I called a taxi, all the way to my front door, and asked to be dropped off at the back entrance to our shared workplace. No cars to knock me down. I would be straight into that taxi, approached from the back. I would ask for the back door of the taxi to adjoin the back door of the workplace. I would give an excuse. Security. And the same when I returned home. Reason over fear. No room for error.

Until the taxi, en route from home to work, came to a halt. On the busy dual carriageway. Something rattling. So Teddy was right. Terrifyingly right. Could I get out and help him identify the noise, asked the driver. No, no, no, I screamed! He looked at me as if I was slightly mad. But this madness had method. To spin the wheel, to save life and limb.

And soon we got going again, me firmly in back seat.

So it was with some surprise, and my almost crazed relief, that we arrived at the door. To park with back door adjoining back door came as a curiosity to the driver. But he nodded sympathetically and I tipped him in thanks.

I skipped up the steps to our plush interconnecting offices, where Teddy wrote software, and I helped him do it. He heard my steps and tried to shut the door, but I was through first. “How are you here?” he shouted. “You’re at home!” Evidently not, I might have replied. Instead, I just stood there, in openmouthed shock at the scene that unfolded before me.

 

Chapter 8
The problem with Teddy

Every drawer had been emptied, every cupboard laid bare, ornaments and accessories opened or turned upside down. If something had been hidden, it would by now have been found. “What is happening?” I would have sat down, but the seats were upturned, and I had no stomach to right them.

“A burglary,” he said. But I didn’t believe him. “Why would burglars turn everything upside down and take nothing?” I asked. “That beggars belief.”

“I disturbed them,” he said, “took about them with my stick. They fled.”

“Let’s call the police,” I insisted, “Check the CCTV.” “No,” he said sharply. “Let’s not.”

A short pause. “Is it safe?” he asked. “Is it safe?”

“Is what safe, Teddy, is what safe?”

He seemed unsure now, what to say or do. “They were my numbers,” he said, “I suggested the numbers. They came up on Saturday night. I know that you keep it here, you always keep your ticket here until you check the numbers on a Wednesday. And I know you never sign it. Be fair, Charlie, let’s share it.”

He looked at me menacingly. Teddy, I knew, was not the sort of man who shared anything. It was all about Teddy. The gift of the thousand pounds now made sense. He had made his case, that I should spin the wheel, that I could re-arrange fate. But a gift so generous. Now I saw. It was his back-up plan.

“No, Teddy, it isn’t safe. I didn’t buy a ticket last week. There’s nothing to share.”

Teddy lunged at me, screaming, before collapsing to the floor, thrashing around. Yet still looking up at me, the look of sheer menace still etched on his face.

I was relieved that I hadn’t bought a ticket. He would have found it, signed it, cashed it, if it had existed. I would not have seen a penny. He had suggested some numbers, but for once this was blind chance. He had not seen the future, the future had grasped him invitingly by the hand. Or so he had thought. And now he sought control, control of what was to come.

I peered now yet further into his soul, and saw it for what it was. I had glimpsed it before. But what I saw now was yet darker. Consciousness without conscience. A man with no love for anything higher or other than himself.

And I saw now how the things he forecast always came true. Because he made them come true. Until now. He was the sort of man who would sell shares in cruise liners and then plant an iceberg, if he could.

“But you would have been rich too!” cried the man who was already rich. The man who lived in a mansion and looked down on the homeless. The man who liked to rip up the charity envelope.

“What shall it profit a man if he should gain the whole world, and lose his own soul?” I asked him now. “Answer me, Teddy!” But no answer came from the man who knew when bad things were going to happen, who knew because he made those things so.

I picked up his stick. I wanted to hit him, to beat him with that shiny, ebony stick. He cowered. A coward, infused with consciousness, but devoid of conscience. I put it down again. It would have given me satisfaction. But it would have made me more like Teddy. For Teddy, his own personal satisfaction was all that mattered.

That was the problem with Teddy. I didn’t want it to be the problem with me.

I sat on the floor, and considered my options.

“I have something to report,” I told the operator. About some bad things that have happened, some things unexplained. Can I speak to the police?”

Bertie’s Big Idea – A Short Story

Bertie’s Big Idea – A Short Story

By Leighton Vaughan Williams

 

Albert ‘Bertie’ Simpson Sinclair was a man who in earlier days might have been described as a bounder and a cad, albeit an immensely likeable and charming member of that sub-species. The problem for Bertie was that he was, as such, a hopeless, if heroic, failure. But Bertie was an optimist, a man who believed in the philosophy of ‘one more push’, of the sure triumph of unsound hope over all too sound experience. And he had an idea which he believed would make him rich. This is the story of Bertie and his magnificent idea.

 

Bertie’s Dream

Mr. Bertie Simpson Sinclair liked to think of himself as an ideas man. And an ideas man he certainly was. He had plenty of ideas, albeit none of them good. But his latest idea was going to be different. Of that he was sure. He had envisaged, in one giant midsummer night’s dream, a scheme to make himself rich, without making others commensurately poor. To this extent, it was an unusual idea for Bertie, for whom all previous schemes consisted of persuading others to part with their money in pursuit of an apparent though negative actual benefit. Bertie called such schemes win-win. By this he meant that he would win twice, first by taking their money, then by virtue of the scheme into which they had invested. The problem for Bertie was that every such scheme remained a dream, for all his boundless wit and charm. Even his plan to sell tips on the horses, then persuade his followers to place their own money on these gems of advice and share with him half the winnings, but none of the losses, failed when faced with the cold light of reality. There were so many others, including Bertie’s ‘Grow rich while you sleep’ manual, his ‘Learn while you doze’ method, his ‘Snooze yourself slim’ prospectus, his ‘Succeed while you slumber’ pamphlet. Bertie reasoned that alert, wakeful people were out of his reach, which left the more reposed segment of the population as his natural target audience. It was not just the fact that he himself was neither rich, learned, successful nor svelte. The real problem for Bertie was that he had singularly failed to convince even one other member of the human race that he could help them become what he so evidently was not. But that, decided Bertie, was about to change. Because of his midsummer night’s dream.

 

Bertie’s Idea

Bertie liked to think of himself as a clubbable man, a sociable ‘bon viveur’ who could mix with natural ease and grace with ladies and gentlemen of refinement. To this end he sought membership of tennis clubs, golf clubs, health clubs, focusing on the most exclusive of each. But Bertie had not grown rich while he slept. On the contrary, he had grown increasingly poor even as he dreamed of growing rich. As such, he was unable to actually gain entry to any of these clubs of the clubbable, as he thought of them. It was all an unrequited dream. But then came the big dream, that midsummer night, the night that inspired Bertie’s big idea. He had dreamed that he was at the door of one of these desirable clubs of the clubbable, begging inwardly to be allowed in, when an elegantly attired gentleman, upon exiting, had spotted the less than svelte figure of the unlearned though charming Bertie, and spoken to him, softly.  “Sir,” he had quietly ventured, “what are you doing waiting at the door? Did you not know that this is a club reserved only for the clubbable?” Taking immediate offence, Bertie’s dreamworld person had risen quickly to his own defence. “I AM a clubbable man,” he had expostulated, invoking his own claims to that most cherished status in society. But something within him had turned, something that was stirred by the well-dressed accuser. And so awoke Bertie, with his brand new big idea, an idea which he had instantly concluded would make him rich.

 

Bertie’s plan

A club for the unclubbable! That’s what he would create. He would create the world’s first club which would only accept members who didn’t want to join, members who were truly unclubbable. He would in other words create a club for those unwilling to join any club that would accept them as a member. The idea was one thing, turning it into a practical scheme was quite another. But that, for Bertie, was the challenge. And the rewards beckoned for Bertie like a shining beacon on a golden hilltop. At least that’s the way that Bertie visualized things. But he knew he was at base camp and the climb that lay ahead was steep and possibly long.

He was not a gifted thinker, but he did have thoughts, and the first of these was to place an advertisement in the local newspaper. Although a man of strictly limited means, it was his only hope of starting the climb which would take him to that shining beacon atop the golden hilltop. The advert was quite simply stated. “Would you join any club that would accept you as a member? If so, we’re wrong for you. We are the world’s only club for the unclubbable. We accept all and only those who don’t wish to join us.”

It was more words than Bertie could really afford, but he had seen that beacon atop the glittering hill and this was his one-time chance to glimpse its light. In the face of that shining lamp, he was steadfast. He would not blink. He waited. For the first response. It arrived by mail the very next day. Addressed to Mr. A. Sinclair, the envelope contained one sheet of blue vellum notepaper. In neat lettering, it was from a Mr. Charles Bone, who simply enquired whether there was an active membership of the club. If so, he was not interested. If not, he might be. Bertie replied with alacrity.  “There is no active membership, so we do not wish to accept you as a member.” By return of post, Mr. Bone accepted membership of this club that didn’t wish to accept him as a member, on one condition. “I am not an active man, and have no wish to be involved with active people. I will join on this condition,” wrote the first and thus far only member of the world’s first club for the unclubbable.

By the same post came an enquiry from a Miss Edith Spratt, who declared herself unwilling to join the club because, while she had been told of the advert, she was not from the local area. As such, she could not make use of its services, even if she wished to, which she did not. Bertie was delighted to accept her as a member, because she was so clearly unable and unwilling to benefit from membership. He wrote to tell her so. On this basis, Miss Spratt became the second member of Sinclair’s club for the unclubbable.

No fee was asked, and none given, by either Mr. Bone or Miss Spratt. But they served their purpose. Neither could in any way reasonably be classed as active members of the fledgling club, but there was now at least a club in existence, and in their different ways both of its members were of the unclubbable kind. There were no further replies to the advertisement, but Bertie was not discouraged. He had left base camp and set forth up the golden hill. He would not turn back.

And so came to Bertie his next idea. If he could introduce Mr. Bone to Miss Spratt, they might help him spread the word through what he conceived as some form of human chain letter that would spread forth and gather together the great unclubbable hordes, brought together into one vast club composed of only those unable and unwilling to join a club.

“Do you possess transport?” Bertie now wrote to Miss Spratt. “Yes”, came the one word reply. Seizing upon this positive news, Bertie devised a plan to bring together the only members of his brand new club. He offered, though he could ill afford it, to pay the cost of fuel for what would be a 70 mile journey for Miss Spratt. The response from Miss Spratt was quick in coming and even quicker in its message. “Dear Mr. Sinclair, my transport is an electric wheelchair. Yours sincerely, Miss E. Spratt”.

To Bertie, that hilltop was starting to look further away than ever.

 

Bertie’s vision

Was Bertie’s vision turning into a mirage? It was a question that might have deterred many, but not a question that deterred Bertie. If Miss Spratt could not come to Mr. Bone, then Mr. Bone must be brought to Miss Spratt, reasoned Bertie with impeccable rigour. Without further ado, he grabbed his quill-like pen, and rushed off a letter. “Dear Mr. Bone, I would like you to meet Miss Edith Spratt.  Like you, she is totally unsuited to the life of a club. In short, she is totally unclubbable. Yet she is a member of the club to which you belong. I think this remarkable coincidence is too great to be overlooked. For that reason, I would like you to meet Miss Spratt. She lives some distance away, but this has the advantage of offering you a pleasant journey even if the meeting is less pleasant than might reasonably be hoped. I hope you reply affirmatively. Yours sincerely, Albert Simpson Sinclair.”

Mr. Bone responded immediately, posing just one short question. “Is Miss Spratt an active member of the club?” Bertie was eager to re-assure. “No, Miss Spratt is not an active member of the club. I trust this reassures you.” It did. The following day, Bertie received the acceptance of his invitation. All that remained was to persuade Edith Spratt to accept the same invitation to meet Mr. Bone. “Dear Miss Spratt,” wrote Bertie, “I would like you to meet Mr. Charles Bone. He is not a clubbable man, and by natural inclination not an active man, but he shares with you membership of the club which I am proud to manage. I trust this remarkable coincidence offers sufficient grounds for you to accept this invitation. Yours sincerely, Albert ‘Please call me Bertie’ Simpson Sinclair.

The letter of response arrived by return of post. Addressed to Mr. Bertie Sinclair, and written in exquisite script, it was simply expressed. “Dear Bertie, I accept your invitation. Please be so kind as to bring Mr. Bone to me. Yours truly, Edith.”

And so was arranged the meeting between Mr. Charles Bone, retired undertaker, and Miss Edith Spratt, lady of leisure, to take place the following Wednesday at the home of Edith Spratt. Thursday and Friday came and went, as did the weekend, but no news leaked out. For several more days, Bertie rushed each morning to pick up the morning mail. But no letter arrived from either Mr. Bone or Miss Spratt. After two weeks had elapsed, which seemed like three months, Bertie reached for his pen and wrote to Mr. Bone. “Dear Mr. Bone, I hope and trust that your meeting with Miss Edith Spratt went well. Perhaps your meeting went so well that you have had little time to write letters. If so, I would be delighted to hear of this happy news, which you might perhaps share much more widely. Yours expectantly, Albert Simpson Sinclair”.

Sooner rather than later a letter arrived, addressed to Mr. A. S. Sinclair.

“Dear Mr. Sinclair,” it read, “Thank you for arranging the meeting between myself and Miss Spratt. You assured me, however, that the lady was not an active member of the club. I cannot agree with your assessment. Could you in future introduce me to one of your less active members? Yours sincerely, Mr. Charles Bone.”

The human chain letter, it seemed, had come apart at the first link.

Bertie took pen to fresh paper, addressed to Miss Edith Spratt.

“Dear Miss Spratt, I understand that no developments arose out of your rendezvous with Mr. Charles Bone, and that you are no longer in contact. Can you confirm my impression? With sincere regards, Albert (Bertie to you) Simpson Sinclair.”

Two days passed, while Bertie fretted. And then it came. The envelope was coloured pink and addressed to Bertie Sinclair. On matching pink notepaper, it simply stated. “Apparently I was too active for the liking of Mr. Bone, or so he told me. Please do, however, feel free to introduce me to someone from your club rather more active than Mr. Bone. Hoping to hear further. Yours in anticipation, Edie.”

 

Bertie’s day

Bertie had lost interest in Mr. Bone, but not in his project. He still possessed the vision of a network of clubs composed entirely of unclubbable people. But the vision was starting, even to Bertie, to flicker a little. His only hope now, he reasoned, lay with Miss Edith Spratt. But he had nobody else to introduce her to, active, inactive or semi-active. Except himself. And so he resolved to visit Miss Spratt at her residence, disguised as a member of his club for the unclubbable. He wrote as follows.

“Dear Edie (if I may), I am sorry to hear that you were too active a member for Mr. Bone. I prefer to see it from a different perspective – that he was not active enough for you. That can easily be remedied. I have on my books a very unclubbable man, who likes his own company, but who I can assure you is a very active member of the club. I will send him to you next Wednesday, if that is convenient. Kindest regards, Bertie.”

Wednesday did prove convenient, and soon a disguised Albert Sinclair, replete with flowing beard, heavy horn-rimmed spectacles and extravagant moustache, was entering the country residence of the wealthy widow newly self-described as Miss Edith Spratt. Introducing himself as Archibald Henry, former solo arctic explorer, he was at once able to tick two boxes, as both a private man and an active man. Miss Spratt was impressed to meet an explorer, less so a former explorer, and even less so a man who had clearly given up the athletic lifestyle at some distant corner in time. They had little in common, so she asked him whether it was cold in the Arctic. Yes, very cold, he said, and there the discussion of his days as an explorer froze. It was only when he spoke of the club that she lit up, asking him whether he had ever met Mr. Bertie Sinclair. She was disappointed to hear he had not, sharing with him her secret crush on this exciting innovator who had created a wonderful club for the unclubbable, and whose charm and good manners flowed out of every word he committed to paper in his delightful letters. She confided in the former explorer how she secretly wished Bertie would visit.

What had he done? This lady of wealth and refinement wanted him, Bertie, and he had entered her life disguised as a hairy arctic explorer. What should he do? Should he discard the disguise and reveal himself, like some sort of superhero, to be the witty, charming man of her dreams? He thought better of it, if only because he wanted more time to think. He bid her farewell and returned the 70 miles to his small suburban bedsit.

He had not spotted the electric wheelchair she had spoken of, but he had been dazzled by the vintage Mercedes sports car gracing her ample driveway. It somehow made her all the more attractive. He slept fitfully that night, rising at dawn to do what he had to.

Drawing from his battered desk the fine stationary he used for only the most important of communications, he applied modern quill to traditional vellum. “Dear Edie,” he wrote, “Mr. Henry, who visited you at my invitation, has contacted me to express his great pleasure at the making of your acquaintance. He tells me, however, that he is not worthy of your notice, and has asked me to convey his great good wishes to you in all you do. Although I am persuaded that I also am not worthy of your notice, I would be happy to follow in the estimable footsteps of our arctic adventurer in order to make your personal acquaintance, should that be your wish. I remain, with the greatest respect, your humble servant. Bertie.”

The next day dragged heavily on Albert Sinclair, as he waited and hoped for a positive reply. He was waiting at the door next day for the arrival of the postman. A quick reply should mean good news, a slow reply worse news, and no reply the worst news of all. The pink envelope arrived at the first opportunity. He opened it gently, hardly daring to read it. “Dear Bertie. I did have some regard for Mr. Archibald Henry, and believed that under his hirsute exterior probably lurked a fine, attractive gentleman. Still, I expect the excess of hair served him well in the cold arctic climate, and he has now grown well accustomed to it. Yes, I would indeed welcome a visit from your fine self. For a man of your considerable talents as gifted entrepreneur, your humility is a further charming sign of the true gentleman that you so clearly show yourself to be. With regards from your friend, Edie.

Bertie could not contain all the excitement shooting through his body. All that stood between him and the wealthy, attractive widow, it appeared, was the removal of his pencil moustache. As such, he would turn up at the elegant doorway, and introduce himself, Albert Simpson Sinclair, to the lady who would clearly not be able to resist his very considerable charms. Wednesday at noon was the agreed time.

 

Bertie’s meeting

She was waiting for him at the door, and extended her hand to him in such a way that he was not sure whether she was expecting him to shake it or kiss it. He shook it. “It is a pleasure and a delight to make your acquaintance in person,” he opened. “Tea or coffee,” she asked. “Coffee, please”. “White or black?” As a man who had not had either tea or coffee made for him for quite some time, he was not used to being questioned about his preferences in such detail. “Black, please, with milk,” he said. She looked at him quizzically. “Yes, plenty of milk,” he confirmed. Decaffeinated, please.

There was no conversation while the coffee was prepared, and after it was served, little more. The series of awkward silences, interrupted by sips of caffeinated coffee, was eventually interrupted by the chime of the grandfather clock standing in the corner of the room, alerting them to the fact that it was 12.30. It presented a much-needed natural break.

“I must take my leave,” said Bertie, “I have so much business to attend to.” There was a further moment of silence, while Miss Spratt rose to her feet, pointing accusingly at him. “What have you done with Bertie?” she asked. “Tell me NOW, what have you done with Bertie?” He was sure he had misheard her. “What have I done with WHAT?” he asked.

“What have you done with Bertie?” she persisted, in an increasingly strident tone. “But I AM Bertie!” “You, Sir, are NOT. You are Mr. Archibald Henry, former arctic explorer. Do you really think you could trick me into thinking you were my Bertie by shaving off your formerly abundant facial hair.” “No,” she continued, “Mr. Archibald Henry minus beard, moustache and large-rimmed spectacles is still Mr. Archibald Henry. Now tell me what you have done with Bertie, or I shall call the police to have you arrested.”

“I AM Albert Simpson Sinclair,” he insisted, “Archibald Henry does not exist.” At these words, Edith Spratt reached urgently for the telephone. “So you are now saying that you, Archibald Henry, do not exist, even though you stand right before me. Is this your defence to the charge of abducting Mr. Sinclair, or worse? A defence of insanity.”

Bertie could see his Big Idea unravelling before his eyes, the dream giving way to stone cold reality. Maybe he was insane, to hope that any idea of his could come true, maybe he was insane to still dream that one day he could persuade people that they could grow rich while they slept, succeed while they slumbered, learn while they dozed, slim while they snoozed. Maybe he was insane to believe that he could create a club for the unclubbable. But he was not insane in the way that Edith Spratt thought he was, and certainly not criminally insane.

For perhaps the first time in many years, he now decided upon a plan at odds with every instinct in his bones, a plan to tell the truth.

“It was I, Albert Simpson Sinclair, who came to your home last week disguised as the fictional arctic explorer, Archibald Henry. It is I, Archibald Simpson Sinclair, who stand before you now. I throw myself upon your good graces. I can do no more.” He paused. “Edie,” he half sobbed now, “I am Bertie.”

Edith Spratt said nothing but put down the telephone she had been wielding with increasing menace. “Mr. Sinclair,” she said quietly. “I am not sure whether you are a good man or a bad man, a sound man or an unsound man, and I am not really concerned to find out.” Bertie winced. “But”, she continued, “I do know the difference between a good idea and a bad idea, a sound idea and an unsound idea. And I am rather attracted to your big idea.” “A club for the unclubbable?” piped up Bertie, excitedly. “Quite so,” declared Miss Edith Spratt. “I shall turn this idea into reality, and because I am a lady of honour and refinement, you shall be rewarded with a respectable share in its fortunes. But be assured, Mr. Sinclair, this shall become my vision, the vision of Edith Evadne Spratt.

And so began a new dawn for Mr. Bertie Sinclair. Employed to use his considerable wit and charm to help expand the Spratt chain of clubs for the unclubbable, his big idea had become reality. He knew now that he would never grow rich while he slept, nor succeed while he slumbered, but he would indeed grow rich, by working hard while awake, and he would succeed. But much more importantly, Mr. Archibald Simpson Sinclair had now achieved a station in life which neither money nor worldly success could alone bestow. Bertie Sinclair, one-time conman, cad and bounder, had been transformed. Eminently clubbable, he had finally become a gentleman.

The Tenth Hole – A Short Story

The Tenth Hole
A short story – based on a real event

By Leighton Vaughan Williams

Based closely on a meeting with a stranger on a golf course, it is a meeting
that has inspired me to write this story. It is a story that I didn’t expect to
need to tell, but have needed to tell. If nobody reads it, no matter. But I
hope people do read it. Especially the people who have lessons still to
learn and dreams still to dream. Especially the people who want to play the
tenth hole! Especially those people. But it is a story for everyone.

Synopsis
It all started on the first green when a little white ball mysteriously appeared over our heads, landing next to the pin. Enter nobody! Then a stranger! The journey had not yet begun but would soon do so. As we parted company at the final green with a firm handshake and a salutary farewell, we realised that we had been part of one of those experiences which challenges one’s fundamental perceptions and pre-conceptions, illuminating us without dazzling us, humbling us without diminishing us. Yet we know not the stranger’s name nor philosophy. We did not ask, nor did we desire to know. We had learned enough without needing to seek further. As I placed my trusty white putter into the bag for the final time in the round, I realised I would never reach for it again in quite the same way. I realised also that our journey could only really be experienced, not told. Yet at the same time I knew that our journey offers a tale that needs to be told and re-told, for as long as there remain in golf, and life, lessons to be learned, dreams to be dreamed. Steve knew so too. He knew so because of what had happened the night before.

Chapter 1
The Third Hole
“Sorry about the shot back there. I didn’t know there was anyone on the green. I should have checked.” In truth, the apologised-for strike up the demanding uphill first hole was not a shot many would have expected to reach the skirts of the putting surface, let alone come to rest blowing a six inch kiss to the flag. I brushed it off with a dismissive wave. It is part of the happenstance that populates any game of golf, and especially one in which the opening challenge is played blind over the brink of a hill peering barely 60 yards to the front edge of the polestick’s personal domain. It was not until we were standing astride the third tee that the stranger had made his presence known, in the guise of one of the series of sauntering singleton players who increasingly grace the summer weekday afternoons with their individual displays of personal golfing panache.

A stranger, a singleton, that much we established on sight. A player of the game whose pace, and possibly skill, somewhat outstripped ours, we now established by simple deduction. “Go ahead of us. Play through.” Steve made the offer, as if in response to the words of apology, though it came out more like a barked command. “But I have all the time in the world.” The stranger’s words could have been spoken with weary regret or mock ribaldry or simply matter-of-factly, but there was no flicker of any of these sentiments. None. They were spoken instead with an inflection of what might be described as detached satisfaction lightly brushed with a hint of passion. Such a vocal inflection was strange, but only because it was unexpected. But it somehow sounded neither odd nor unusual. Nor was it discomforting. Yet we were discomforted, for another reason. That offer to go ahead of us, such a superficially selfless act of generosity, was in fact neither selfless nor generous. It was born rather out of a simple desire to see the stranger proceed on his way. And to free us from the burden of a pressing presence to our rear. As well as potential exposure to a dimpled projectile rising bullet-like in undetected flight toward our unprotected frames. And to free us from the burden of unbidden external inspection and inner judgement. We played on, in the quiet hope, even expectation, that a couple of wayward and horizontally challenged strikes off the tee would achieve the result which invitation had regrettably failed to achieve. The concept of limitless time would be put to the test of the real ticking clock.

I struck the ball first, to see it simply disappear from sight. In my experience this always means one thing – that the ball, while technically still in its own existence, would never again be part of mine. It was lost. Steve kept track of his own ball, but only because it had travelled a bare 30 yards towards its destination roughly a quarter of a mile away. We looked to the stranger. “Bad luck” , he intoned. “It happens to us all.” “Play through?”, I enquired. The stranger seemed to consider for an extended moment. “I know where your ball is. I’ll show you.” The ball was soon retrieved, from some of the rough grass designed to challenge players on the adjacent hole. Meanwhile, Steve had played his second shot. Eventually we reached the putting green. As we considered line and length, a ball could be seen descending from its high trajectory. Struck from the third tee, it landed short of our feet but long of reason. An obvious fluke, this was a ball delivered to its flight path as if by mechanical, not human, means. Inspired, we sank our putts. We bagged our respective putters, one made for the left handed and one for the right, and turned, a little lighter of foot, to tee off at the short fourth. Our path meandered edgingly close to the little white missile which awaited the stranger’s next smack. We paused as he came closer. “That was a big hit”, I ventured. “It achieved its purpose,” he smiled. “After all, you sunk your putts.” I considered how he knew our green play from so afar, but set the thought aside. “By the way, thanks for finding my ball. I was sure it was lost.” He looked at me quizzically. “If you know where something is, can it be lost?” He paused for a moment. “I’m not just talking about golf, you know. I’m really not!”

Chapter 2
The Fourth Hole
It is a relatively short hole, so the course designers had felt the need to compensate. They dug a deep bunker to the front side of the green, another to the right side and a steep grassy bank running down its back side. But we were confident. That’s what a hole-dropping long putt does for you. But the tee is set atop a flight of short steps, offering a balcony view to any short strike to the third. So we viewed. The stranger obliged. He knew where the hole was. And he knew how to reach it. Two feet from the hole. Now a simple tap-in, he marked the ball, cleaned it, set it down again. He tapped but it stayed out. He turned and looked up to us atop our temporary perch. He was smiling broadly. He was happy. Now it was my turn. To be happy. To sidestep or overfly sandy trap, to shy short of flirtation with treacherous back side.

I visualised. I swung golf’s version of bat at ball. I succeeded. In my own way. I was satisfied. But I was not happy. Not happy like the stranger. As Steve stepped up, the happy face, now muted, stood with him, still, close yet distant. Club hit ball, ball responded. Then misbehaved. “Difficult hole.” Steve and I were silent, then Steve spoke. “We’re not on song today. You go ahead. Please proceed.” Proceed he did, to place tee in ground, ball on tee, administer smart blow of short club to small ball. Perfect shot, perfect line, perfect distance, if aimed at the deep forward sandy trap. “Beached!”, he noted. It was a simple statement of fact. “I could be quite some time. You two play ahead.” We did.

Steve withdrew a club of middling length and swung freely at the ball. A pleasing crack. It was well timed. We all knew it. It found the green, albeit the right edge, opposite end to the stick. But nice enough. This was a turnaround. Both Steve and I with at least a chance of potting the hole in three, and the stranger’s ball buried deep below a gaping, uninviting, curling upper lip. This was an unequal contest between skill and natural sand, tipped heavily in favour of nature. And tipped in favour of us. I had clear sight of the pin, about 40 feet away, angled just forward of my left shoulder. Maybe 12 feet to the relatively smooth surface which surrounded the flag. I was furthest from the hole, if measured by distance. I locked my wrists and applied blade to ball. Ball bounced away to order. Slow surface. Good for once. Just short and below the hole. Steve was on the green. Still separated, ball to cup, by more than the stranger. But Steve’s ball had the advantage. It could see the objective. So could Steve. He took his time. He took his time on that putt as if he had all the time in the world. He had the line. He sort of had the length. Came up a nose short, queried its options, then decided to drop. Three. Par. He seemed to wave. Almost imperceptibly, but he seemed to wave. He seemed to wave to the stranger.

Now it was my turn. A five footer to share the hole. Don’t leave it short. Don’t leave it short. I didn’t. Hole shared. Move on or wait for splash of grainy sand. We waited and got a shower, and a show, as the flecked sphere soared from its tomb aloft a chariot of spray. It was long of the pin, before the backspin, and less long after, but still the length of a longshot. “You go ahead,” he shouted. “I might be quite some time.”

Chapter 3
The Fifth Hole
We walked on to the difficult fifth. My honour again to tee off first. 480 yards to the middle of the green. No need to guess. Modern technology has seen to that. Smooth swing. This time it flew straight, but not particularly long, and so tracked by eye through its entire flight. “I’m happy enough with that”. Could have been longer but a shot to make you happy. Now Steve inserted his tee into the sloping ground, placed his ball on it and prepared to play. “Same action, using a higher tee, and you would get the same result and a lot more length.” The stranger spoke softly but it interrupted Steve’s pre-shot ritual. Steve stopped and took a few steps backward. Hesitant now, where he had been confident, he sort of tiptoed to tee, imparted a dull thwack to ball and stood back to inspect the damage. He had topped it, had hit just its very top. It had at least gone straight.

It was in the short grass, that much could be said for it. But if only he had cradled the ball upon a higher tee. He remembered the stranger’s words. He looked at the stranger. The stranger said nothing. He had no need to. Steve was still a good 400 yards short of the green, I a little closer, but there was someone with us who placed his ball on a somewhat higher tee than either of us. I saw an opportunity. “Why don’t you show us how you tee up your ball, and show us the result?” He obliged, propelling the ball into an unusually steep trajectory, unusual by our standards.

“How did you do that?”, I asked. “Practice,” he murmured. “And belief.”
“So you can do it every time?” I was curious. “Not at all,” he replied.”You saw. I just missed a simple putt back there. It happens.” He paused. “Acceptance. When we fail we need acceptance. It’s about belief and acceptance. And practice.”
“Easier when the putts are dropping, and you’re hitting the greens. It’s when they’re not that the doubt creeps in.” I looked at Steve, still mulling over his mishit.
The stranger smiled, as if he’d heard it all before. “That, of course, is when your belief in your game is most important of all. Trust me!”

I did trust in something. I trusted in his own belief in his game. I shared his belief in his game. It was my game that I didn’t trust, that I didn’t believe in. I told him just that. “Well, it’s a start,” he said. “It’s a good start.” And he was off. “See you!” We bid the stranger well as he strode off over the artificial horizon created by the dip down the hill. “Try transferring some weight onto your back foot if you want lift.” He was calling back to us. “But don’t expect miracles. Genuine miracles are rare.”

Soon out of sight, he was not out of mind. We played on. To play safe or straddle the lake? Doubt or belief? We both chose the middle path, left of lake and far short of target. And found, in each case, the creek. Doubt played no further part. Lost ball, dropped ball, lose a shot, carry on. Lost ball? “It’s not lost if you know where to find it!” It was the same voice, of the stranger. “But I don’t know where to find it.” “Nor I”, echoed Steve. “Here they are!” Balls both found, thanks to stranger. We could barely see them, but there they were, buried treacherously, all but invisibly deep. How did he spot them? Why was he looking? Questions soon rendered academic. We could not retrieve them. Thank you stranger, we thought, you have shown us what we looked for. We know more but have gained nothing. But we were thinking of golf. We were only thinking of golf. We had learned nothing. But the stranger, he wasn’t just talking about golf, you know. He really wasn’t! We didn’t mind that. Get a new ball out of our bags. Time to play our shots. So we did. We were playing golf. And the game, to win or lose,was on. So to the green.

Preparing to putt for the hole. I’m confident. I bring the putter head straight back. Now don’t decelerate on the shot! A cry from the direction of adjacent tee. ‘Fore!’ “Watch out!” I react, instinctively, and miss. In truth, the rocketing wayward ball missed my person by all of six feet. “Unlucky,” Steve commiserated. But he was happy really. He shared the hole. I was lucky in another sense. Six feet to the left and I might have played my last shot. I hadn’t prepared for that. You never do. You’re concentrating too much on playing the game.

Chapter 4
The Sixth Hole
I like the sixth hole, except for the tree, the tree that always grabs my ball out of its soaring greenward flight. A neighbouring golf club had taken a vote on whether to demolish a tree. Their golfers had the same problem. It stopped the ball going where it was hit. It was a hazard, like a sand trap, only taller and more explicitly menacing. They voted to chop it down. Ours is much the same, but there’s no plan for a vote. One of my balls is still up it, trapped for posterity. I guess it will still be there when I am gone. I think about that every time I pass that tree. It was there as I considered my second shot. We had both hit pretty standard drives up the hill, off the tee. If the ball flies far enough that it disappears over the brow of the hill, I always jump a little. I don’t actually leave the ground, but I jump inside, if you understand. I suppose Steve does too, though not when I do it. Then he has the opposite feeling. There’s not much room for empathy on the golf course. Maybe there should be. It’s only a game, they say. But they’re wrong. Games are important. They teach you things. About yourself. And about others.

Standing atop the crest of the hill, and a few strides beyond, I prepare to avoid the tree. It’s to my left, about 100 yards, maybe a bit more. Should be able to avoid it this time. Hit straight or perhaps a little right. Then it’s a safe, controlled punch to the middle of the green. I can dream. That’s part of the game. But do I believe? I think I do. Then I see the branches. I always see the branches. I try to see the smooth, even grass of the fairway beyond the tree. I believe I will reach it. I’m sure I will reach it. But what about the branches? It’s always about the branches. There are so many branches. I prepare to play. A shout to me, this time heralding advice not danger. “It’s not about the tree!” It was the stranger. “It really isn’t about the tree. It’s about the fairway. Focus on that and you’ll be safe.” Easy for him to say that, I thought. He knows the game. He may as well have invented it. But I didn’t know the game. Well, I knew the rules, at least the ones that let me play the game, but I didn’t really know the game. Not like the stranger. “Trust me!” I looked at him, bag of clubs slung over his right shoulder, left hand propped against tree. “Keep standing there and it’s you who’ll be needing to trust,” I shouted back. Addressing the ball, I took aim. He didn’t take cover. He seemed to know. As did I. I simply knew that I wasn’t going to hit that tree. I trusted him. Despite the branches.

As I walked past the tree toward the pleasant fairway lie, I thanked him. I didn’t even ask him why he had waited, and not moved on to the next hole. I didn’t need to. And I didn’t want to. He returned the thanks. “You won’t always miss that tree,” he said. “But you will always know that you can.” He walked on to the next hole. My next shot found the deep rough, then the sand, then three more shots. Steve raised his hand. But he didn’t jump. Not in the air. Just inside. So did I. For a different reason, so did I.

Chapter 5
The Seventh Hole
Hit it down the fairway. Straight as you can. Easy shot to the green. One putt. In the hole. Shout of ‘Birdie.’ Job done. Before we hit the ball, we can all dream. I had the dream that day. Until I swung the club. The ball bore no blame. It had no will, no self-control. It simply obeyed instructions. But I had will, I had the freedom to instruct it as I liked. And I had the dream. So why was the ball buried deep in the bushes? I had teed it up high as I should, swung club with smoothness of action, completed follow through with self-referenced elegance. So why? I could have asked the stranger, had he been there. Steve was there, but he didn’t know the answer, even if it had been a real question.

Then I saw a dog. Looked like a nice dog, a spaniel, I think. I don’t know a lot about dogs so I can’t be sure. I just know that there are nice dogs and nasty dogs. This one looked nice. The one that bit Steve a few weeks back was the other kind. That dog wasn’t on a lead. It was free to bite and it bit. It bit Steve on the belly. But he didn’t tell anyone, not anyone in authority. That was his choice. He chose to hope it wouldn’t do it again. Steve wasn’t doing anything wrong, not when he was bit. He had no choice about that. The choice came later. I had a choice. I could have chosen to give the ball better instructions. But it was not a real choice. I did try. But now I had a real choice. To forget the fluffed shot, drop another ball, lose a shot, address the ball, play as if this was the ball’s natural resting place. Or to continue asking ‘why’ in plaintive cry. I knew what the stranger would do. So I played the ball, as best I could. Not in anger but in thanks. Thanks to the stranger. And now to the lady with the dog, who shouts “Well played!” The kindness of strangers. Quite pleased with the outcome. Not as pleased as Steve who will soon be at the flag. Or at least near enough to ensure the hole. But I was playing against the course. When Steve plays well, I play against the course. When Steve plays badly, I play against Steve. Confident now, unhurried, undisturbed. I dream of holing it in one from here. Hit the flag. That’ll do. I’ll be happy with that. I look around. Nobody to impress this time.

The lady has moved on, taking her nice dog with her. It had no choice. It was on a lead. The man in the motorised grass-cutter does have a choice. He can choose to buzz around when you are trying to play, or buzz off until you’ve played. He chooses to buzz around. He always does. It stops him getting bored. He’s near enough now that the ball might hit him if you take your shot quickly. The stranger could definitely hit him. He has the skill. But you’d have to be lucky. He’s not even wearing a helmet. His own silly fault if I hit him. I feel lucky. Then I see him. Back of the green. Watching. I see the stranger. Embarrassed, I aim for the green, and almost hit the flag. I’m playing the game now. That’s why games are important. They teach you things. Steve wins the hole, but I’ve won too. The man in the machine has moved on. So has the stranger. And so, I feel, have I.

Chapter 6
The Eighth Hole
The weather forecast had said heavy showers. But I didn’t believe the forecast. They say bad things and if they’re wrong, you will be happy anyway. And forget their mistake. But if they say good weather, and you are drenched, you will not easily forgive them. Clever strategy. But no good if you want to know how the weather will be. So I didn’t believe, and persuaded Steve I was right. He believed me. Now he was wet. I saw the lady with the dog in the distance. She had opened an umbrella. I wondered about the stranger. Now we had a decision to make. We could take whatever partial shelter we could or play on and be fully sheltered sooner. We played on.

The weather did its worst but only served to help us focus. Two shots. Straight off the tee. Both off the putting green. But neither by much. Then a flash. Count to three and the crash of thunder. Not far away now. I know someone who knew someone who was struck by lightning. The girl she knew was on a golf course when it happened. Did everything right. Made all the right choices once she saw the electric fork. But she hadn’t believed the forecast. Clever strategy, she had said, to forecast storms. You’d forgive and forget if the weather was good. This time it wasn’t. The forecasters were right. She was wrong. She paid the price. A high price for getting it wrong, but life is like that sometimes. This was on my mind as I lined up my shot, designed as a little chip to the flag. Maybe my wrist quivered. I suppose the ball obeyed my instructions, it always does, but not my intent. Short, very short of the pin. Steve wasn’t scared like me. He said that it was very long odds to be struck dead. But he didn’t know someone who knew someone who succumbed despite the odds. I think it makes a difference. At least it made a difference that day. He holed it. And would probably live to tell the tale. Very probably, according to the odds. We move on to the closing hole of this nine hole course. You can play eighteen holes if you like, but not eighteen different holes. You just play the nine holes twice. We play them just once. For us, there is no tenth hole.

Chapter 7
The Ninth Hole
The heavens continued to flash, almost in time now with thunderous clash, but Steve was unconcerned. He was chatting with someone in what the British call a golf buggy and the Americans call a cart. Steve climbed in. Then climbed out. “Safer in there”, he said. But he was wrong. First advice is to get clear of open frame vehicles. I told him. “But they’re earthed, grounded. Look at those rubber tyres.” Steve was sure of his physics. I didn’t know the physics but I knew the advice. Get clear, it said. “Anyway, meet Chris, who knows about these things.” My heart sank. The stranger, in whom I’d placed so much trust. Sitting in the nippy four-wheeler. So sure of himself. Placing his trust in a death trap. Now it made sense. I thought as we left the fourth green that I’d seen Steve wave at him. Almost imperceptibly, but definitely an acknowledgment. I had put it out of my mind, put it down to courtesy, but now it all came together. He was not a stranger, not to Steve. I went over to Chris, back turned but obviously impervious to danger. Blissful in ignorance. I tapped on the shoulder of the white collared shirt. Chris turned. She turned and looked at me. “Always best to hire the car when there’s a chance of thunder”. It was one of the local lady golfers, good player, better than us, at golf, but no better at physics. I laughed. Out of relief, really. The stranger would never sit in an open vehicle in a thunderstorm. We were free to do what we wanted, but not the stranger. He would only do what was right. I trusted in him. Then I saw him, at the crest of the hill, over which we hoped to propel our first volleys.

He was calling us on. I felt safe now. And Steve seemed to wave. He seemed to wave to the stranger. We cleared the hill, first bounce I prefer to recall, and took good advantage of the downward sloping smooth grassed fairway. Two further blows apiece, of club on ball, and we were on the green. Then two putts from Steve, one from me. “Well played, Sirs!” He joined us on the green, shook our hands firmly. “The game is important,” he said. It teaches us things, about ourselves, and about others. The question is how we play it.” We wanted to play it like him.

On to the tenth hole?” he asked. “There is no tenth hole,” we said, almost in unison. “There are only the ones you see. You keep going round and round. Until you stop. That’s all there is. Didn’t you know?” The stranger smiled. He knew that we were still talking about golf. “There is a tenth hole,” he said. But you need to look for it. And to believe in it.” He proceeded to hand each of us a little white ball. They were the same two balls, with their distinctive markings, that we had lost in the creek. I was momentarily staggered. “But how? They were lost, unreachable.” “Yes, those balls truly were lost, they were unreachable,”‘said the stranger. “But on the tenth hole, there are no lost balls. Nothing is lost on the tenth. Trust me.”

We did trust him now. We had seen and we believed. “You remember that shot off the tee, the one that soared so high, over the very tall pines. I saw you watching. Well, you can do that too. You can do that, when you reach the tenth hole.”
Then, with a wave, he started to walk away. “It’s about belief and acceptance”, he called back, “and practice. So, practise these things.” The stranger now out of sight, Steve turned to me. “He gave me a lift home last night,” he said, “when I missed the last bus, and was caught three miles from home in that terrible storm. He stopped his car and offered me a lift. Went totally out of his way, took me to my front door, then he was gone. I didn’t even have chance to thank him.”

“So why didn’t you say anything before?” I was perplexed. “Because I didn’t recognise him at first, though I did feel that I knew him. I didn’t recognise him until he called us on in the storm.” “Steve, are you absolutely sure it was him?” “Not for certain,” he said, “But that doesn’t really matter, does it?” “I suppose not,” I agreed. “I suppose not.” The kindness of strangers. What a wonderful thing. We looked to the sky. The clouds were slowly parting. We knew where we were now. We knew now that there was a tenth hole!

Strictly Come Dancing: the luck of the draw really does matter!

Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.

A viscountess, a radio DJ, a reality star, a vlogger, a comedian, several sportspeople and an assortment of actors and presenters. These, more or less, are the celebrities lined up to compete in the 2019 season of Strictly Come Dancing.

Outside their day jobs, few people know much about them yet. But over the 13 weeks or so of shows up until Christmas, viewers will at least learn how well the contestants can dance. But how much will their success in the competition have to do with their foxtrot and to what extent will it be, literally, the luck of the draw that sees the victors lift the trophy in December?

seminal study published in 2010 looked at public voting at the end of episodes of the various Idol television pop singing contests and found that singers who were later on in the bill got a disproportionately higher share of the public vote than those who had preceded them.

This was explained as a “recency effect” – meaning that those performing later are more recent in the memory of people who were judging or voting. Interestingly, a different study, of wine tasting, suggested that there is also a significant “primacy effect” which favours the wines that people taste first (as well, to some extent, as last).

A little bias is in order

What would happen if the evaluation of each performance was carried out immediately after each performance instead of at the end – surely this would eliminate the benefit of going last as there would be equal recency in each case? The problem in implementing this is that the public need to see all the performers before they can choose which of them deserves their vote.

Dress rehearsal for Strictly Come Dancing, August 2019. madathanu / Shutterstock.com

You might think the solution is to award a vote to each performer immediately after each performance – by complementing the public vote with the scores of a panel of expert judges. And, of course, Strictly Come Dancing (or Dancing with the Stars if you are in the US) does just this. So there should be no “recency effect” in the expert voting – because the next performer does not take to the stage until the previous performer has been scored.

We might expect in this case that the later performers taking to the dance floor should have no advantage over earlier performing contestants in the expert evaluations – and, in particular, there should be no “last dance” advantage.

We decided to test this out using a large data set of every performance ever danced on the UK and US versions of the show – going right back to the debut show in 2004. Our findings, published in Economics Letters, proved not only surprising, but almost a bit shocking.

Last shall be first

Contrary to expectations, we found the same sequence order bias by the expert panel judges – who voted after each act – as by the general public, voting after all performances had concluded.

We applied a range of statistical tests to allow for the difference in quality of the various performers and as a result we were able to exclude quality as a reason for getting high marks. This worked for all but the opening spot of the night, which we found was generally filled by one of the better performers.

So the findings matched the Idol study in demonstrating that the last dance slot should be most coveted, but that the first to perform also scored better than expected. This resembles a J-curve where there are sequence order effects such that the first and later performing contestants disproportionately gained higher expert panel scores.

Although we believe the production team’s choice of opening performance may play a role in this, our best explanation of the key sequence biases is as a type of “grade inflation” in the expert panel’s scoring. In particular, we interpret the “order” effect as deriving from studio audience pressure – a little like the published evidence of unconscious bias exhibited by referees in response to spectator pressure. The influence on the judges of increasing studio acclaim and euphoria as the contest progresses to a conclusion is likely to be further exacerbated by the proximity of the judges to the audience.

When the votes from the general public augment the expert panel scores – as is the case in Strictly Come Dancing – the biases observed in the expert panel scores are amplified. All of which means that, based on past series, the best place to perform is last and second is the least successful place to perform.

The implications of this are worrying if they spill over into the real world. Is there an advantage in going last (or first) into the interview room for a job – even if the applicants are evaluated between interviews? The same effects could have implications in so many situations, such as sitting down in a dentist’s chair or doctor’s surgery, appearing in front of a magistrate or having your examination script marked by someone with a huge pile of work to get through.

One study, reported in the New York Times in 2011, found that experienced parole judges granted freedom about 65% of the time to the first prisoner to appear before them on a given day, and the first after lunch – but to almost nobody by the end of a morning session.

So our research confirms what has long been suspected – that the order in which performers (and quite possibly interviewees) appear can make a big difference. So it’s now time to look more carefully at the potential dangers this can pose more generally for people’s daily lives, and what we can do to best address the problem.

Why do we always seem to just miss the bus? It’s nothing to do with luck.

Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.

The bus arrives every twenty minutes on average, though sometimes the interval between buses is a bit longer and sometimes a bit shorter. Still, it’s 20 minutes taken as an average, or an average of three buses an hour. So you emerge onto the main road from a side lane at some random time, and come straight upon the bus stop. How long can you expect to wait on average for the next bus to arrive?
The intuitive answer is 10 minutes, since this is exactly half way along the average interval between buses, and if your usual wait is rather longer than this, then you have been unlucky.
But is this right? The Inspection Paradox suggests that in most circumstances you will actually be quite lucky only to wait ten minutes for the next bus to arrive.
Let’s examine this more closely. The bus arrives every 20 minutes on average, or three times an hour on average. But that is only an average. If they actually do arrive at exactly 20 minute intervals, then your expected wait is indeed 10 minutes (the mid-point of the interval between the bus arrivals). But if there is any variation around that average, things change, for the worse.

Say for example, that half the time the buses arrive at a ten minute interval and half the time at a 30 minute interval. The overall average is now 20 minutes, but from your point of view it is three times more likely that you’ll turn up during the 30 minute interval than during the ten minute interval. Your appearance at the stop is random, and as such is more likely to take place during a long interval between two buses arriving than during a short interval. It is like randomly throwing a dart at a timeline 30 minutes long. You could well hit the ten minute interval but it is much more likely that you will hit the 30 minute interval.
So let’s see what this means for our expected wait time. If you randomly arrive during the long (30 minute) interval, you can expect to wait 15 minutes. If you randomly arrive during the short (10 minute) interval, you can expect to wait 5 minutes. But there is three times the chance you will arrive during the long interval, and therefore three times the chance of waiting 15 minutes as five minutes. So you expected wait is 3×15 minutes plus 1x 5 minutes, divided by four. This equals 50 divided by 4 or 12.5 minutes.
In conclusion, the buses arrive on average every 20 minutes but your expected wait time is not half of that (10 minutes) but more in every case except when the buses arrive at exact 20 minute intervals. The greater the dispersion around the average, the greater the amount by which your expected wait time exceeds the average wait time. This is the ‘Inspection Paradox’, which states than whenever you ‘inspect’ a process you are likely to find that things take (or last) longer than their ‘uninspected’ average. What seems like the persistence of bad luck is actually the laws of probability and statistics playing out their natural course.
Once made aware of the paradox, it seems to appear everywhere.
For example, take the case where the average class size at an institution is 30 students. If you decide to interview random students from the institution, and ask them how big is their class size, you will usually obtain an average rather higher than 30. Let’s take a stylised example to explain why. Say that the institution has class sizes of either ten or 50, and there are equal numbers of both class sizes. So the overall average class size is 30. But in selecting a random student, it is five times more likely that he or she will come from a class of 50 students than of ten students. So for every one student who replies ‘10’ to your enquiry about their class size, there will be five who answer ’50.’ So the average class size thrown up by your survey is 5×50 + 1 x 10, divided by 6. This equals 260/6 = 43.3. So the act of inspecting the class sizes actually increases the average obtained compared to the uninspected average. The only circumstance in which the inspected and uninspected average coincides is when every class size is equal.
The range of real-life cases where this occurs is almost boundless. For example, you visit the gym at a random time of day and ask a random sample of those who are there how long they normally exercise for. The answer you obtain will likely well exceed the average of all those who attend the gym that day because it is more likely that when you turn up you will come across those who exercise for a long time than a short time.

Once you know about the Inspection Paradox, the world and our perception of our place in it, is never quite the same again.

Exercise

You arrive at someone’s home and are ushered into the garden. You know that a train passes the end of the garden every half an hour on average but the trains are actually scheduled so that half pass by with an  interval of a quarter of an hour and half with an interval of 45 minutes. Given that you have no clue when the last train passed by and the scheduled interval between that train and the next, how long can you expect to wait for the next train?

Links and References

Amir D. Aczel. Chance: A Guide to Gambling, Love, the Stock market and Just About Everything Else. 18 May, 2016. NY: Thunder’s Mouth Press.

On the Persistence of Bad Luck (and Good). Amir Aczel. Sept. 4, 2013. http://blogs.discovermagazine.com/crux/2013/09/04/on-the-persistence-of-bad-luck-and-good/#.XXJL0ihKh3g

The Waiting Time Paradox, or, Why is My Bus Always Late? https://jakevdp.github.io/blog/2018/09/13/waiting-time-paradox/

Probably Overthinking It. August 18, 2015. The Inspection Paradox is Everywhere. http://allendowney.blogspot.com/2015/08/the-inspection-paradox-is-everywhere.html

When the world’s greatest diarist wrote to the world’s greatest scientist.

Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.

One of the most celebrated pieces of correspondence in the history of probability and gambling, and one of which I am particularly fond, involves an exchange of letters between the greatest diarist of all time, Samuel Pepys, and the greatest scientist of all time, Sir Isaac Newton.

The six letters exchanged between Pepys in London and Newton in Cambridge related to a problem posed to Newton by Pepys about gambling odds. The interchange took place between November 22 and December 23, 1693. The ostensible reason for Mr. Pepys’ interest was to encourage the thirst for truth of his young friend, Mr. Smith. Whether Sir Isaac believed that tale or not we shall never know. The real reason, however, was later revealed in a letter written to a confidante by Pepys indicating that he himself was about to stake 10 pounds, a considerable sum in 1693, on such a bet. Now we’re talking!

The first letter to Newton introduced Mr. Smith as a fellow with a “general reputation…in this towne (inferiour to none, but superiour to most) for his maistery [of]…Arithmetick”.

What emerged has come down to us as the aptly named Newton-Pepys problem.

Essentially, the question came down to this:

Which of the following three propositions has the greatest chance of success.

  1. Six fair dice are tossed independently and at least one ‘6’ appears
  2. 12 fair dice are tossed independently and at least two ‘6’s appear.
  3. 18 fair dice are tossed independently and at least three ‘6’s appear.

Pepys was convinced that C. had the highest probability and asked Newton to confirm this.

Newton chose A as the highest probability, then B, then C, and produced his calculations for Pepys, who wouldn’t accept them.

So who was right? Newton or Pepys?

Well, let’s see.

The first problem is the easiest to solve.

What is the probability of A?

Probability that one toss of a coin produces a ‘6’ = 1/6

So probability that one toss of a coin does not produce a ‘6’ = 5/6

So probability that six independent tosses of a coin produces no ‘6’ = (5/6)6

So probability of AT LEAST one ‘6’ in 6 tosses = 1 – (5/6)6 = 0.6651

So far, so good.

The probability of problem B and probability of problem C are more difficult to calculate and involve use of the binomial distribution, though Newton derived the answers from first principles, by his method of ‘Progressions’.

Both methods give the same answer, but using the more modern binomial distribution is easier.

So let’s do it, along the way by introducing the idea of so-called ‘Bernoulli trials’.

The nice thing about a Bernoulli trial is that it has only two possible outcomes.

Each outcome can be framed as a ‘yes’ or ‘no’ question (success or failure).

Let probability of success = p.

Let probability of failure = 1-p.

Each trial is independent of the others and the probability of the two outcomes remains constant for every trial.

An example is tossing a coin. Will it lands heads?

Another example is rolling a die. Will it come up ‘6’?

Yes = success (S); No = failure (F).

Let probability of success, P (S) = p; probability of failure, P (F) = 1-p.

So the question: How many Bernoulli trials are needed to get to the first success?

This is straightforward, as the only way to need exactly five trials, for example, is to begin with four failures, i.e. FFFFS.

Probability of this = (1-p) (1-p) (1-p) (1-p) p = (1-p)4 p

Similarly, the only way to need exactly six trials is to begin with five failures, i.e. FFFFFS.

Probability of this = (1-p) (1-p) (1-p) (1-p) (1-p) p = (1-p)5 p

More generally, the probability that success starts on trial number n =

(1-p)n-1 p

This is a geometric distribution. This distribution deals with the number of trials required for a single success.

But what is the chance that the first success takes AT LEAST some number of trials, say 12 trials?

One method is to add the probability of 12 trials to prob. of 13 trials to prob. of 14 trials to prob. of 15 trials, etc.  …………………………

Easier method: The only time you will need at least 12 trials is when the first 11 trials are all failures, i.e. (1-p)11

In a sequence of Bernoulli trials, the probability that the first success takes at least n trials is (1-p)n-1

Let’s take a couple of examples.

Probability that the first success (heads on coin toss) takes at least three trials (tosses of the coin)= (1-0.5)2 = 0.25

Probability that the first success (heads on coin toss) takes at least four trials (tosses of the coin)= (1-0.5)3 = 0.125

But so far we have only learned how to calculate the probability of one success in so many trials.

What if we want to know the probability of two, or three, or however many successes?

To take an example, what is the probability of exactly two ‘6’s in five throws of the die?

To determine this, we need to calculate the number of ways two ‘6’s can occur in five throws of the die, and multiply that by the probability of each of these ways occurring.

So, probability = number of ways something can occur multiplied by probability of each way occurring.

How many ways can we throw two ‘6’s in five throws of the die?

Where S = Success in throwing a ‘6’, F = Fail in throwing a ‘6’, we have:

SSFFF; SFSFF; SFFSF; SFFFS; FSSFF; FSFSF; FSFFS; FFSSF; FFSFS; FFFSS

So there are 10 ways of throwing two ‘6’s in five throws of the dice.

More formally, we are seeking to calculate how many ways 2 things can be chosen from 5. This is known as ‘5 Choose 2’, written as:

5 C 2= 10

More generally, the number of ways k things can be chosen from n is:

nC k = n! / (n-k)! k!

n! (known as n factorial) = n (n-1) (n-2) … 1

k! (known as k factorial) = k (k-1) (k-2) … 1

Thus, 5C 2 = 5! / 3! 2! = 5x4x3x2x1 / (3x2x1x2x1) = 5×4/(2×1) = 20/2=10

So what is the probability of throwing exactly two ‘6’s in five throws of the die, in each of these ten cases? p is the probability of success. 1-p is the probability of failure.

In each case, the probability = p.p.(1-p).(1-p).(1-p)

= p2 (1-p)3

Since there are 5 C 2 such sequences, the probability of exactly 2 ‘6’s =

10 p2 (1-p)3

Generally, in a fixed sequence of n Bernoulli trials, the probability of exactly r successes is:

nC r x pr (1-p) n-r

This is the binomial distribution. Note that it requires that the probability of success on each trial be constant. It also requires only two possible outcomes.

So, for example, what is the chance of exactly 3 heads when a fair coin is tossed 5 times?

5C 3 x (1/2)3 x (1/2)2 = 10/32 = 5/16

And what is the chance of exactly 2 sixes when a fair die is rolled five times?

5 C 2x (1/6)2 x (5/6)3 = 10 x 1/36 x 125/216 = 1250/7776 = 0.1608

So let’s now use the binomial distribution to solve the Newton-Pepys problem.

  1. What is the probability of obtaining at least one six with 6 dice?
  2. What is the probability of obtaining at least two sixes with 12 dice?
  3. What is the probability of obtaining at least three sizes with 18 dice?

First, what is the probability of no sixes with 6 dice?

P (no sixes with six dice) = n C x . (1/6)x . (5/6)n-x, x = 0,1,2,…,n

Where x is the number of successes.

So, probability of no successes (no sixes) with 6 dice =

n!/(n-k)!k! = 6!/(6-0)!0! x (1/6)0 . (5/6)6-0 = 6!/6! X 1 x 1 x (5/6)6 = (5/6)6

Note that: 0! = 1

Here’s the proof: n! = n. (n-1)!

At n=1, 1! = 1. (1-1)!

So 1 = 0!

So, where x is the number of sixes, probability of at least one six is equal to ‘1’ minus the probability of no sixes, which can be written as:

P (x≥ 1) = 1 – P(x=0) = 1 – (5/6)6 = 0.665 (to three decimal  places).

i.e. probability of at least one six = 1 minus the probability of no sixes.

That is a formal solution to Part 1 of the Newton-Pepys Problem.

Now on to Part 2.

Probability of at least two sixes with 12 dice is equal to ‘1’ minus the probability of no sixes minus the probability of exactly one six.

This can be written as:

P (x≥2) = 1 – P(x=0) – P(x=1)

P(x=0) in 12 throws of the dice = (5/6)12

P (x=1) in 12 throws of the dice = 12 C 1 . (1/6)1 . (5/6)11nC k = n! / (n-k)! k!

So 12 C 1

= 12! / (12-1)! 1! = 12! / 11! 1! = 12

So, P (x≥2) = 1 – (5/6)12 – 12. (1/6) . (5/6)11

= 1 – 0.112156654 – 2 . (0.134587985) = 0.887843346 – 0.26917597 =

= 0.618667376 = 0.619 (to 3 decimal places)

This is a formal solution to Part 2 of the Newton-Pepys Problem.

Now on to Part 3.

Probability of at least three sixes with 18 dice is equal to ‘1’ minus the probability of no sixes minus the probability of exactly one six minus the probability of at exactly two sixes.

This can be written as:

P (x≥3) = 1 – P(x=0) – P(x=1) – P(x=2)

P(x=0) in 18 throws of the dice = (5/6)18

P (x=1) in 18 throws of the dice = 18 C 1 . (1/6)1 . (5/6)17

nC k = n! / (n-k)! k!

So 18 C 1

= 18! / (18-1)! 1! = 18

So P (x=1) = 18.  (1/6)1 . (5/6)17

P (x=2) = 18 C 2 . (1/6)2 .(5/6)16

18 C 2

     = 18! / (18-2)! 2! = 18!/16! 2! = 18. (17/2)

So P (x=2) = 18. (17/2) (1/6)2 (5/6)16

So P(x=3) = 1 – P (x=0) – (P(x=1) – P (x=2)

P (x=0) = (5/6)18

= 0.0375610365

P (x=1) = 18. 1/6. (0.0450732438) = 0.135219731

P (x=2) = 18. (17/2) (1/36) (0.0540878926) = 0.229873544

So P(x=3) = 1 – 0.0375610365 – 0.135219731 – 0.229873544 =

P(x≥3) = 0.597345689 = 0.597 (to 3 decimal places, )

This is a formal solution to Part 3 of the Newton-Pepys Problem.

So, to re-state the Newton-Pepys problem.

Which of the following three propositions has the greatest chance of success?

  1. Six fair dice are tossed independently and at least one ‘6’ appears.
  2. 12 fair dice are tossed independently and at least two ‘6’s appear.
  3. 18 fair dice are tossed independently and at least three ‘6’s appear.

Pepys was convinced that C. had the highest probability and asked Newton to confirm this.

Newton chose A, then B, then C, and produced his calculations for Pepys, who wouldn’t accept them.

So who was right? Newton or Pepys?

According to our calculations, what is the probability of A? 0.665

What is the probability of B? 0.619

What is the probability of C? 0.597

So Sir Isaac’s solution was right. Samuel Pepys was wrong, a wrong compounded by refusing to accept Newton’s solution. How much he lost gambling on his misjudgement is mired in the mists of history. The Newton-Pepys Problem is not, and continues to tease our brains to this very day.

References and Links

Newton and Pepys. DataGenetics. http://datagenetics.com/blog/february12014/index.html

Newton-Pepys problem. Wikipedia. https://en.wikipedia.org/wiki/Newton%E2%80%93Pepys_problem